


CONFIGURATION EVALUATION AND CRITERIA PLAN 


VOLUME 2 - EVALUATION CRITERIA PLAN 
(PRELIMINARY) 


Space Transportation Main Engine 
(STME) Configuration Study 


Contract NAS8-36867 
Data Requirement DR-9 
August 1986 


Prepared For: 


NASA George C. Marshall Space Flight Center 
Marshall Space Flight Center, Alabama 35812 


Prepared By 



,K. Bair 
Study Manager 


Approved By: 



Program Manager 


Aerojet TechSy stems Company 
P.O. Box 13222 

Sacramento, California 95813 


RPT/BB0323 



FOREWORD 


This is the Evaluation Criteria Plan for the Space Transportation Main 
Engine Configuration Study and has been prepared as part of Task 3.0 of 
Contract NAS8-36867. The work is being performed by the Aerojet TechSystems 
Company for the NASA - Marshall Space Flight Center. 

The program objective is to identify candidate main engine configurations 
which enhance launch vehicle performance, operation and cost. These candidate 
configurations will be evaluated and the conf iguration(s) which provide signi- 
ficant advantages over existing systems will be selected for consideration for 
the next generation launch vehicles. 

The NASA-MSFC Project Manager is Mr. N. Hughes. The ATC Program Manager 
is Mr. D.C. Rousar and the ATC Study Manager is Mr. E.K. Bair. 

The Evaluation Criteria Plan is Volume 2 of the Configuration Evaluation 
and Criteria Plan, Contract Data Requirement DR-9. Volume 1 is the System 
Trades Study and Design Methodology Plan. 


TABLE OF CONTENTS 


Page 


I. 

Introduction 

1 

II. 

Evaluation Criteria 

5 

III. 

Yes/No Evaluation Criteria 

10 

IV. 

Quantitative Criteria Weighting 

12 

V. 

Baseline Engine 

26 


iv 


LIST OF TABLES 

Table No. Page 


I Quantitative Evaluation Criteria 7 
II Category Cost and Weighting 13 
III Turnaround and Unscheduled Activity Evaluation 19 


V 


LIST OF FIGURES 


Figure No. Page 


1 Configuration Evaluation and Selection Plan 2 

2 Candidate Engine Screening Process 3 

3 Yes/No Screening Process 11 

4 Candidate Engine Evaluation Format 15 

5 Performance Criteria Evaluation 17 

6 Operation and Support Evaluation 18 

7 Development Evaluation 21 

8 Production Criteria Evaluation 23 

9 Availability Criteria Evaluation 24 

10 Facilities Criteria Evaluation 25 


I. INTRODUCTION 


The unbiased selection of the Space Transportation Main Engine (STME) 
configuration requires that the candidate engines be evaluated against a 
predetermined set of criteria which must be properly weighted to emphasize 
critical requirements defined prior to the actual evaluation. Since the 
optimum configuration is a compromise between engine and airframe design, the 
criteria and relative weighting of the criteria involves a team effort between 
ATC, MSFC and the Space Transportation Architecture Study (STAS) contractors. 
The evaluation and selection process, Fig. 1, involves the following func- 
tions: (1) determining if a configuration can satisfy basic STME requirements 
(yes/no) (2) defining the evaluation criteria, (3) selecting the criteria 
relative importance or weighting, (4) determining the weighting sensitivities 
and (5) establishing a baseline for engine evaluation. The criteria weighting 
and sensitivities are cost related and are based on mission models and vehicle 
requirements. The criteria, weighting and sensitivity will be reviewed for 
concurrence by MSFC prior to conducting the evaluation process. 

The evaluation process is used as a coarse screen to determine the can- 
didate engines for the Task 3 Parametric studies and as a fine screen to 
determine concept(s) for conceptual design. Task 4. The criteria used for the 
coarse and fine screen evaluation process is shown in Figure 2. 

The coarse screen process involves verifying that the candidate engines 
can meet the "yes/no" screening requirements and a semi-subjective quanti- 
tative evaluation. 

The fine screen engines have to meet all of the "yes/no" screening gates 
and are then subjected to a detailed evaluation or assessment using the quan- 
titative cost evaluation processes. The option exists for re-cycling a con- 
cept through the quantitative portion of the screening and allows for some 
degree of optimization. 

The basic vehicle is a two stage LOX/HC, LOX/LH 2 parallel burn vehicle 
capable of placing 150,000 lbs in low earth orbit (LEO). The mission model 
calls for placement of 800 payloads in LEO starting in the 1995 to 2000 time 
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Figure 1. Configuration Evaluation and Selection Plan 
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I, Introduction (cont.) 


frame. The 800 missions are to be accomplished with 8 vehicles (100 missions 
each). 

The basic STME is conceived to be a L0X/LH2* 580K (vacuum) thrust engine 
with a burn time of 520 seconds and a total life of 100 missions. 
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II. EVALUATION CRITERIA 


The evaluation criteria define the significant functions that are 
required to properly evaluate an engine system. These criteria include all 
the significant items covered by the STAS studies in the architecture evalua- 
tion as well as items considered significant by ATC. The criteria must allow 
evaluation from both an engine and vehicle system point of view for proper 
integration into a complete system. Concepts must pass an initial "yes/no" 
screening and then are subjected to a quantitative evaluation based on cost. 

A. Yes/No Screen Evaluation 

Initially, an engine concept must pass an evaluation relative to 
"yes/no" type criteria. The concept must be judged as a “yes" in all areas in 
order to be given further consideration. These criteria are; 

Safety 

Maximum Envelope 

Gimballing Capability 

Sea Level and Altitude Start Capability 

Single or Multi Engine Application 

Expendable or Reusable 

Throttling Capability 

One Hundred Mission Capability 

IOC Compatibility 

A positive answer must be given for all of the listed criteria; if it cannot 
pass these criteria then it has failed to meet the fundamental requirements 
set down for the STME. 

B. Quantitative (Cost Based) Screen Evaluation 

The quantitative criteria screen evaluation is based on cost and is 
divided into the following six categories: (1) Performance and weight. 
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II, Evaluation Criteria (cont.) 


(2) Operation and Support, (3) Development, (4) Production, (5) Availability 
and (6) Facilities. Each category is then further subdivided into subcriteria 
to allow for a quantitative evaluation (Table I). 

Performance and Weight 

The performance and weight category includes specific impulse and engine 
weight. The specific impulse represents a major factor in engine cost and 
complexity and vehicle system design. The engine propellant, efficiency, 
cycle, mixture ratio and chamber pressure are the primary factors in determin- 
ing specific impulse. 

Engine weight is dependent on thrust and chamber pressure requirements. 
The weight is not a totally dominant factor in vehicle design and the airframe 
contractors are willing to sacrifice some engine weight to enhance operations, 
reliability and life. 

Operation and Support 

The operation and support category includes the criteria involved with 
defining the operations cost of an engine. This includes life and maintain- 
ability. 

Engine life is dependent on cycle selection and chamber pressure. Higher 
chamber pressures require higher pump discharge pressures, coolant pressure 
drop and heat fluxes making it more difficult to maintain life margins. 

Maintainability contains the operations and support costs, including 
turnaround activities, unscheduled maintenance and overhaul requirements. 

Turnaround activities include the functions necessary to ready an engine 
system from flight to flight. This activity is the major contributor to the 
operation and support category. The following activities are considered in 
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TABLE I 


C ategory 

Performance and Weight 
Operation & Support 

Development 

Production 

Availability 

Facilities 


Quantitative Evaluation Criteria 
Criteria 


Isp 

Weight 

Maintainability 
Safety & Reliability 


Development Time, 
Risk, and Cost 


Mission Availability 

Development 

Launch/GSE 


Subcriteria 


Turnaround 

Unscheduled 

Overhaul 


Component 

Engine 
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II, Evaluation Criteria (cont.) 


the turnaround evaluation: leak checks, diagnostics, horoscope inspections, 
pump torque checks, injector inspection, gas generator/preburner inspection, 
turbine system inspection and coolant system inspection. 

Unscheduled maintenance is dependent on relative engine complexity and 
cycle selection. Some configurations can lead to increased susceptibility to 
component replacements prior to engine overhaul. 

Engine overhaul is dependent on the engine cycle selection and configu- 
ration. Engine cycle and chamber cooling methods determine the extent of 
inspection and replacement requirements. 

Reliability is dependent upon the engine's ability to meet its perfor- 
mance and operational design requirements. It is the composite result of 
establishing adequate design margins and the ability to predict the engine 
loading and operating environments. 

Development 

The development category includes the factors that determine the DOT & E 
costs for developing an engine system. Engine cycle, thrust level, chamber 
pressure, propellant selection and life are important factors in determining 
development costs. Additionally, technology availability and development risk 
must be considered in this category. Only engines which can be developed in 
the required time will be given further consideration. 

Production 


The production category includes the factors that determine the produc- 
tion cost of an engine system. Engine cycle, thrust level, chamber pressure 
and the number of engines produced are used to determine production costs. 
The number of engines produced is also a function of engine life. 
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II, Evaluation Criteria (cont.) 


Availability 

Availability is a reflection of the engine system's reliability and is 
considered in this evaluation as a measure its ability to meet the mission 
needs on time. An availability value of 98% was used in establishing this 
criteria. 

Facilities 


The facilities category determines the development and launch/ground 
support requirements for the engine development, acceptance and use. The 
development facilities are dependent on engine cycle propellants and chamber 
pressure. The launch and ground support criteria is dependent on propellant 
selections and engine cycle. 
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III. “YES/NO" EVALUATION CRITERIA 


As discussed in Section II, the initial engine evaluation is a "yes/no" 
screening which requires that a concept pass every element in the criteria to 
be considered further. The process is shown graphically in Fig. 3. 
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Figure 3. “Yes/No” Screening Process 
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IV. QUANTITATIVE CRITERIA WEIGHTING 


The mission model assumed for use in developing the quantitative evalua- 
tion criteria uses an eight vehicle fleet, each capable of 100 missions. The 
first mission occurs between 1995 and 2000. For the upper stage, two 580,000 
Ibf (vacuum) engines are assumed with a burn time totalling 520 seconds for 
each mission. 

For use in the evaluations, it was assumed that combined capability of 
existing NSTL and other (new or existing) facilities would be in place and 
operation in time to support the STME development, qualification and produc- 
tion acceptance test schedule demands. 

The quantitative criteria weighting represents the relative importance of 
the defined categories and criteria used to evaluate an engine system. The 
STAS contractors recoimnend costing the criteria categories to establish their 
relative importance, and this is the procedure which will be used. The cost 
values for each category are shown in Table II and were estimated, in part, 
using ATC's Cost Estimating Relationships (CER) developed for the STAS 
contractors with the NASA-MSFC supplied mission model. The STAS equations 
were used for development and production cost estimates. The performance and 
weight criteria are based on the effect of Isp and engine weight. The 
performance and weight cost is determined based on the potential loss of 
revenue due to a lower performing engine (-10 sec max) and the impact of added 
engine weight (1600 lbs, max, total). Because of added propellant or engine 
weight, a corresponding payload loss is incurred with an attendant loss in 
revenue. This is applied across the entire mission model. The costs 
represent the worst case. 

The operations and support evaluation criteria are based upon scheduled 
and unscheduled maintenance costs and overhaul costs as reflected by life and 
start capability and reliability/cost effects. The scheduled and unscheduled 
maintenance costs reflect Space Shuttle derived values while the overhaul 
value is based on historic Titan data. Reliability, relative to meeting 
operational goals, is treated as a bias on the total (sum) operation costs. 
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TABLE II 


Category Cost* and Weighting 


Category 

Cost 

Weighting 
Value (Rounded) 

Performance and Weight 

3.6 

Billion 

25% 

Operation and Support 

5.4 

Bill ion 

40% 

Development 

3.0 

Billion 

20% 

Production 

.7 

Billion 

5% 

Avail abi 1 ity 

1.2 

Billion 

8% 

Facilities 

.2 

Billion 

2% 


14.1 

Billion 

100% 


* Cost values reflect worst case engine 
(Scores zero in weighting value analysis) 
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IV, Quantitative Criteria Weighting (cont.) 


i.e., a .9 operations reliability will impose an additional (1-R)=10% on the 
operations costs. The facility costs were estimated assuming the construction 
of five engine test stands, an engine component test facility for pump, cham- 
ber and GC/preburner development and GSE requirements. The availability 
category was estimated using the probability that the launch system could 
successfully meet its schedule and delivery requirements. 

After developing a cost for each category as discussed above, a percen- 
tage of the total cost was assigned to each category. The individual category 
costs and weighting are shown on Table II. Each of the criteria and subcri- 
teria was similarly subdivided based on cost. The nominal value for each 
criteria is shown in Figure 4 which also shows the format for quantitative 
evaluation of the individual criteria and engine concepts. 

Once the criteria weighting is established, the sensitivity of the cri- 
teria is required. The sensitivity is the unit change per unit criteria 
weight. The first step is to determine the total range of variation in the 
criteria for the engines considered. This range is then evaluated against the 
criteria weight. The ratio of range/wt is the sensitivity to variations. A 
linear variation in weight is assumed for the range of each criteria con- 
sidered. 

Performance 


The weighting for the performance criteria is based on the impact the 
variation in performance will have on payload delivery capability. In this 
case, a total performance variation of 10 seconds of Isp was assumed. Using a 
modified aV requirement, which accounts for gravity, drag and thrust losses, 
and an assumed lift-off thrust to weight ratio of 1.3, the propellant differ- 
ence imposed by the 10 second performance variations was determined. This was 
converted to equivalent payload assuming a $500/1 b to LEO delivery cost. 
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Figure 4. Candidate Engine Evaiuation Format 













IV, Quantitative Criteria Weighting (cont.) 


The cost effect of engine weight was based on a potential total engine 
weight variance of 1600 lbs. This was then equated to loss of payload. 

The effect of performance and weight variation was found to amount to 25% 
of the total weighting score (100), with Isp performance being 21% and weight 
being 4%. The performance evaluation criteria is graphically shown in Fig. 5 

Operation and Support 

The weighting for the operation and support category, shown in Figure 6, 
is based on historic Space Shuttle data for the SSME for scheduled and 
unscheduled maintenance and Titan for overhaul. 

The normal turnaround maintenance costs were determined to have a weight- 
ing value of 10 and unscheduled maintenance, 18. Using historic Titan data on 
overhaul (complete) resulted in a weighting of 8 for this category. The 
overhaul value is based on a twenty-five mission between overhaul basis. 

The reliability is based on a lower limit of .9. This value is based 
upon the engine ability to meet operational needs and is not meant to reflect 
reliability relative to structural failures. 

The Operation and Support Evaluation is shown in Fig. 6 and a breakdown 
of turnaround and unscheduled maintenance item U shown in Table III. 

Development 

The weighting for the development criteria is based on the STME DDT & E 
cost equations for developing a 580K Ibf thrust engine. The cost equations 
use propellant, thrust and chamber pressure for cost estimates. The maximum 
DOT & E cost for a 580K STME is approximately $3 billion which equates to a 
criteria weight of 20. 
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Evaluation Criteria 


Maximum 

Weighting 

Value 


Operation 

and 

Support 

Maintainability 

Turnaround 

10 

Unscheduled 

18 

Overhaul 

8 

Reliability 


4 





Overhauls Required 



Legend 
L Low 
M Medium 
H High 


Figure 6. Operation and Support Evaluation 
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TABLE III 


Turnaround and Unscheduled Activity Evaluation 




Task Complexity 

Turn Around Activity 

Max. Value 

VL L M H 

Leak 

2 


Diagnostic 

2 


Boroscope 

2 


Torque 

2 


Cleaning 

2 

To 


Unscheduled Activity 

Remove/ Install Engine 

5 


Inspect for Foreign Objects 

5 


Di scol orat i on/D i stort i on 

4 


Damage Evaluation/Repair 

4 

T8 



Legend 

VL - Very Low 5 pts 
L - Low 3 pts 
M - Medium 2 pts 
H - High 1 pt 
VH - Very High 0 pts 


IV, Quantitative Criteria Weighting (cont.) 


Each engine configuration will be cost estimated using the basic equa- 
tions supplied to the STAS contractors. 

The STAS equation for a cryogenic propellant engine for DDT & E is based 
on historic data and is expressed as; 

3.7274 X 10^ 11 + 5 X 10"® (Pc-lOOO)] T*®°® 

For a 580K, 3200 psi Pc engine, the basic DDT & E is estimated to be 
$2.4 X 10^. This value is then modified to account for required technology 
gains and a risk assessment. 

The technology gain is based on an advancement from a technology level of 
4 (Critical Function/Characteristics Demonstrated) to a level 6 
(Prototype/Engineering Model Testing in Relevant Environment) and the risk 
level is assumed to be medium (6): (1) Technology exists but has never been 
demonstrated, (2) Alternatives are possible but are costly in terms of 
dollars, and (3) Resources and schedules are marginal for parallel development 
but parallel developments are still possible. The resulting multiplier on 
this basic is approximately 1.25 making the maximum expected DDT & E $ 3 X 

q 

10’, with a weighting value of 20. 

The development evaluation criteria is shown in Fig. 7. 

Production 


The production costs are based on a total engine assembly demand of 
twenty engines. 


Flight; 8 Vehicles, 2 Engines/Vehicle = 16 
2 Qualification Engines = 2 

2 Spare Engines = 2 

Total = 20 


(Vehicle System) 
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Figure 7. Development Evaluation 
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IV, Quantitative Criteria Weighting (cont.) 


Since the SSME is probably close to the STME in size its approximate produc- 
tion cost was used to establish the upper end of the STME production cost. 

The SSME production cost per engine was set at $35 X 10®. The resulting STME 
production cost was found to be $.7 X 10^ which has a weighting value of 5. 

The production evaluation relationship is shown in Figure 8. 

Availability 

The availability criteria is based on the assumption that the vehicle, 
based on STME availability, can meet the launch schedule a minimum of 98% of 
the time. The 2% unavailability is a loss and equates to a weighting value of 
8 when considered as loss of revenue. The availability evaluation criteria is 
shown in Fig. 9. 

Facilities 


The weighting for the facility category is based on the cost of the new 
facilities involved with the development and launch support of the STME. 

These costs are estimated to be approximately $200 million which would yield a 
relative weight of 2. 

The development facilities are divided into two areas component and 
engine. The engines are rated on the level of component testing that can be 
accomplished and on the complexity of the engine testing. Gas generator 
cycles are ranked higher than staged combustion cycles for component level 
testing because the hardware can be tested at the component or subassembly 
level prior to engine level testing. Also, a configuration that allows for 
component level testing and maturity prior to engine level testing would be 
ranked higher than systems that had little maturity prior to engine level 
testing. 

The launch and ground support criteria is a function of the propellant 
handling requirements and system complexity. Fig. 10. 
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Figure 8. Production Criteria Evaluation 
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Figure 9. Availability Criteria Evaluation 
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Figure 10. Facilities Criteria Evaluation 


25 



V. BASELINE ENGINE 


The optimum engine is represented by the addition of the weighting values 
for each criteria, which totals 100 points maximum. This represents an engine 
design which provides the highest performance for the lowest price with the 
lowest operation and support costs. Rating the engines against the maximum 
value for a given category results in obtaining a clear perspective on the 
relative strengths and weaknesses of a given candidate configuration. 


